Image capturing apparatus and control method therefor

ABSTRACT

An image capturing apparatus comprises an image sensor including a photoelectric conversion portion configured to receive light beams having passed through a portion of a pupil area of an imaging lens and output an image signal, a control unit configured to output, from the photoelectric conversion portion, a plurality of image signals captured by shifting a focus position of the imaging lens, an acquisition unit configured to acquire an image shift amount on an image sensing plane of the image sensor, which corresponds to a shift amount when shifting the focus position of the imaging lens; and a calculation unit configured to calculate a correction coefficient for shading on the image sensing plane based on the image shift amount by comparing signals corresponding to the same object among the plurality of image signals.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image capturing apparatus which usesan image sensor having a pupil-dividing function and, more particularly,to a technique of satisfactorily correcting shading.

2. Description of the Related Art

There are conventionally proposed an image capturing apparatus forpupil-dividing one imaging optical system using a relay lens, andperforming exposure using two image sensors, and a method of obtainingpupil-divided images by dividing a photoelectric conversion portionbelow a microlens into a plurality of portions. These apparatus andmethod can be used to, for example, obtain a stereo image using theparallaxes of a plurality of obtained images, perform automatic focusdetection using a phase difference detection method, and create adistance image.

To obtain a distance image or perform focus detection, it is necessaryto know how much two images shift in a pupil-divided direction, and thusa correlation operation such as SAD or SSD is performed. In acorrelation operation, since the shift amount between images having ahigh degree of matching is searched for, if the degree of matching ofthe images with a correct shift amount is low, it is difficult to detecta correct shift amount. A factor which decreases the degree of matchingof the images can be a difference in shading. It is, therefore, desiredto satisfactorily correct shading.

For example, Japanese Patent No. 2715958 discloses a technique ofestimating a shading level by detecting a level difference between an Aimage and a B image based on an image shift amount obtained by acorrelation operation, and calculating a shading correction coefficient,thereby performing shading correction. Japanese Patent No. 4265029discloses a technique of obtaining exit pupil information from aninterchangeable lens, and calculating a shading correction coefficient.

In the above-described technique disclosed in Japanese Patent No.2715958, however, since a correlation operation is performed in advance,if a wrong image shift amount is detected by the first correlationoperation, a shading correction amount is largely wrong. Even though thecorrelation operation is wrong, its reliability improves.

Furthermore, in the above-described technique disclosed in JapanesePatent No. 4265029, since it is necessary to perform geometriccalculation, the calculation amount is large, and it takes too long toperform calculation.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of theabove-described problems, and provides an image capturing apparatuswhich can calculate a shading correction coefficient with a smallcalculation amount without performing a correlation operation inadvance.

According to the first aspect of the present invention, there isprovided an image capturing apparatus comprising: an image sensorincluding a photoelectric conversion portion configured to receive lightbeams having passed through a portion of a pupil area of an imaging lensand output an image signal; a control unit configured to output, fromthe photoelectric conversion portion, a plurality of image signalscaptured by shifting a focus position of the imaging lens; anacquisition unit configured to acquire an image shift amount on an imagesensing plane of the image sensor, which corresponds to a shift amountwhen shifting the focus position of the imaging lens; and a calculationunit configured to calculate a correction coefficient for shading on theimage sensing plane based on the image shift amount by comparing signalscorresponding to the same object among the plurality of image signals.

Furthermore, according to the second aspect of the present invention,there is provided a control method for an image capturing apparatuswhich includes an image sensor having a photoelectric conversion portionconfigured to receive light beams having passed through a portion of apupil area of an imaging lens and output an image signal, the methodcomprising the steps of: causing a control unit to output, from thephotoelectric conversion portion, a plurality of image signals capturedby shifting a focus position of the imaging lens; causing an acquisitionunit to acquire an image shift amount on an image sensing plane of theimage sensor, which corresponds to a shift amount when shifting thefocus position of the imaging lens; and causing a calculation unit tocalculate a correction coefficient for shading on the image sensingplane based on the image shift amount by comparing signals correspondingto the same object among the plurality of image signals.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the arrangement of an image capturingapparatus according to the first embodiment of the present invention;

FIGS. 2A and 2B are views showing the pixel structure of an imagesensor;

FIG. 3 is a view showing the pixel array of the image sensor;

FIG. 4 is a view showing the relationship between pupil division and afocus position;

FIGS. 5A and 5B are views showing a vignetting shape depending on animage height and lens frames;

FIG. 6 is a view showing the relationship between the vignetting shapeand a divided pixel;

FIG. 7 is a view showing the relationship between a screen position anda vignetting state;

FIGS. 8A and 8B are views showing the relationship between shading andan image capturing signal;

FIG. 9 is a view showing the relationship between a focusing lensposition and an image shift position;

FIG. 10 is a view showing the relationship between an image shift andshading;

FIG. 11 is an internal block diagram showing a shading correctioncoefficient calculation unit;

FIG. 12 is a flowchart illustrating the overall operation of the imagecapturing apparatus;

FIGS. 13A to 13D are views each showing the pixel structure of an imagesensor according to the second embodiment of the present invention;

FIG. 14 is a block diagram showing the arrangement of an image capturingapparatus according to the second embodiment of the present invention;and

FIG. 15 is a view showing the arrangement of a relay lens.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will be described in detail belowwith reference to the accompanying drawings.

First Embodiment

FIG. 1 is a block diagram showing the arrangement of an image capturingapparatus according to the first embodiment of the present invention.FIGS. 2A and 2B are views showing the pixel structure of an image sensor102 shown in FIG. 1. FIG. 2A is a view showing the pixel when viewedfrom above. FIG. 2B is a sectional view showing the pixel. Referring toFIGS. 2A and 2B, reference numeral 201 denotes a microlens; and 202 and203, photoelectric conversion portions. In a general image sensor, onephotoelectric conversion portion is arranged for one microlens. However,in the image sensor of the first embodiment, a photoelectric conversionportion is divided into two portions each of which can read outdifferent signals. With the structure shown in FIGS. 2A and 2B, it ispossible to obtain pupil-divided images.

FIG. 3 is a view showing the array of color filters. This array iscalled a Bayer array in which R (red), G (green), and B (blue) filtersare two-dimensionally arranged at a given repetition period. The arrayshown in FIG. 3 makes it possible to create a color video signal from asignal obtained by adding the pixel values of the photoelectricconversion portions 202 and 203. In this embodiment, a color videosignal is created by adding the signals of the photoelectric conversionportions 202 and 203.

The overall arrangement of the image capturing apparatus according tothe first embodiment will be described with reference to FIG. 1.Referring to FIG. 1, reference numeral 101 denotes an imaging lens; 102,the image sensor; 103, an A/D converter; and 104, an AB image separator.The image sensor alternately outputs the signal of the firstphotoelectric conversion portion 202 for receiving light having passedthrough a first pupil area of the imaging lens 101 and that of thesecond photoelectric conversion portion 203 for receiving light havingpassed through a second pupil area of the imaging lens 101. The functionof the AB image separator 104 separates two signals, which have beenpupil-divided by the photoelectric conversion portions 202 and 203, intoA and B image signals for the respective pupil areas. The A and B imagesignals output from the AB image separator 104 are input to shadingcorrection units 106 and 107, and corrected by them, respectively. Notethat the B image signal is also transferred to a shading correctioncoefficient calculation unit 108. The shading correction coefficientcalculation unit 108 outputs A and B gain images, which are input to theshading correction units 106 and 107, respectively.

The A and B image signals which have been corrected by the shadingcorrection units 106 and 107 are input to a focus detection unit 110 andan addition Bayer forming unit 111. The addition Bayer forming unit 111adds the A and B images, and a video signal processing unit 112 convertsthe resultant image into a video signal. The output of the video signalprocessing unit 112 is used for display and recording. Display andrecording are irrelevant to the present invention and a detaileddescription thereof will be omitted. A microcomputer 109 is a portionfor controlling the overall image capturing apparatus.

The characteristics of the pupil-divided images will be described withreference to FIG. 4. Reference numeral 601 denotes a lens. Exposuresignals divided and input to photoelectric conversion portions 202 and203 have characteristics equal to those of signals having passed throughexit pupils 602 and 603. With respect to light beams passing through theexit pupils 602 and 603, light beams emitted from one point of an objectconverge on one point again on a focus plane 605 as a plane which is infocus. The light beams emitted from one point of the object areprojected at different positions on a front plane 604 or rear plane 606with reference to the focus plane 605. If the light beams exiting fromthe same position of the object pass through the exit pupils 602 and 603and their projection positions shift from each other, this indicates anout-of-focus state. It is, therefore, possible to calculate a defocusamount based on the shift amount between the A and B images.

The focus detection unit 110 shown in FIG. 1 detects the shift amountbetween the A and B images. If there is a level difference between the Aand B images due to shading when detecting the shift amount between theimages, an error may occur in image shift detection or detection mayfail. It is, therefore, necessary to correct shading.

The reason why shading occurs will be described with reference to a lenssectional view shown in FIG. 5A. Referring to FIGS. 5A and 5B, referencenumeral 701 denotes a front lens; 702, a stop; and 703, a rear lens. Theframe of the front lens 701 will be referred to as a front framehereinafter, and a frame formed by the rear lens 703 will be referred toas a rear frame hereinafter. Reference numeral 704 denotes an imagesensing plane. FIG. 5B shows overlapping of the frames of the front lens701, stop 702, and rear lens 703 when viewed from a position x on theimage sensing plane 704, and overlapping of the frames of the front lens701, stop 702, and rear lens 703 when viewed from a position y on theimage sensing plane. When viewed from the position x, only the stoplimits the amount of light. However, when viewed from the position y,the front frame 701 and rear frame 703 also limit the amount of light.

FIG. 6 shows overlapping of a range within which light reaches and thephotoelectric conversion portions of the image sensor when viewed fromthe position y shown in FIGS. 5A and 5B. The photoelectric conversionportions 202 and 203 have largely different ranges within which lightreaches. Shading indicates a phenomenon in which the amount of lightdecreases as the image height increases away from the center of theoptical axis. The pupil-divided images have a property that the balancebetween the A and B images is lost as the image height increases. Theshape of a hatched portion shown in FIG. 6 will be referred to as avignetting shape, and FIG. 7 shows the relationship between thevignetting shape and a pixel position on the sensor. The vignettingshape varies depending on a location, and gradually changes as thelocation shifts.

FIG. 8A is a view in which the abscissa represents a position on theimage sensor 102 and the ordinate represents the amount of light. Theshape shown in FIG. 8A indicates a shading image. By capturing a whiteuniform surface, it is possible to obtain an image matching the shadingimage shown in FIG. 8A. Unless a white uniform surface is captured, itis impossible to obtain a shading image. In general, a video signal isobtained by superimposition of the shading image and the image of theobject, as shown in FIG. 8B. In this embodiment, a shading image isobtained from the video signal obtained by superimposition of theshading image and the image of the object, thereby obtaining a shadingcorrection coefficient.

The principle of this embodiment will be explained with reference toFIG. 9. Although the focus plane has been described with reference toFIG. 4, the focusing lens is generally moved to match the focus planewith the image sensing plane. The focus plane 605 is moved by moving thefocusing lens to match the plane of the image sensor 102.

In FIG. 9, reference numeral 1105 denotes a plane of the image sensor.When the focusing lens is at a position as indicated by referencenumeral 1101, an image having passed through the pupil 602 is at aposition on the image sensing plane as indicated by reference numeral1103. To the contrary, if the focusing lens is at a position asindicated by reference numeral 1102, the image moves to a position asindicated by reference numeral 1104. That is, it is indicated that it ispossible to acquire images when the same object is projected atdifferent positions on the image sensing plane. In this embodiment, ashading correction coefficient is obtained using a plurality of imagescorresponding to the different focusing lens positions. Since themovement amount of the image with respect to the movement amount of thefocusing lens is a constant amount determined by a baseline length, aposition to which the image has moved can be known by only multiplyingthe lens movement amount by a constant coefficient.

FIG. 10 is an enlarged view showing a portion 901, shown in FIG. 8B, ofthe signals 1103 and 1104. The signals 1103 and 1104 shown in FIG. 10are obtained by enlarging portions of the video signals of the sameobject exposed at different focusing lens positions. A line connectingcorresponding points of the signals matches a shading curve 902. It is,therefore, possible to obtain a shading curve by comparing the videosignals of the same object at locations having a horizontal shift amountdetermined based on a focusing positional difference. Note that sinceshading decreases a transmittance, it is necessary to obtain a levelratio rather than a difference. When the focusing lens is moved by asmall amount, the image also moves by only a small amount. It is,however, possible to obtain a shading curve for the entire area bycontinuously connecting the obtained ratios.

FIG. 11 is an internal block diagram showing details of the shadingcorrection coefficient calculation unit 108 shown in FIG. 1. The shadingcorrection coefficient calculation unit 108 is finely controlled by themicrocomputer 109, and a timing generator 509 precisely controlsinternal circuits.

An input signal is only a B image, and time-divisionally exposed Bimages corresponding to different focusing lens positions arerespectively stored in a first image memory 501 and second image memory502. Low-pass filters 503 and 504 remove the high frequency componentsof the first and second images, and a delay circuit 506 delays only thesecond image by a predetermined amount. The images input to a divider507 for obtaining a ratio horizontally shift due to the effect of thedelay circuit 506. The amount by which the delay circuit 506 delays theimage is determined based on its baseline length.

The output of the divider 507 is a transmittance ratio with a pixel at adistance of a pitch determined based on the baseline length. A shadingcoefficient image memory 508 can accumulate correction coefficients forone screen. When correction coefficients for the entire screen areaccumulated, a normalization circuit 510 reads out a coefficient,normalizes it to a gain with a magnification of 1 at the center of theimage height, and writes back the result in the shading coefficientimage memory 508. After that, A and B gain images are read out at atiming according to an overall synchronization signal in synchronismwith the image sensor. Since the input first and second images are Bimages, the A gain image is generated by addressing the memory so as tobe the inverted image of the B gain image.

Control of the overall image capturing apparatus by the microcomputer109 will be described with reference to a flowchart shown in FIG. 12.

In step S1201, a system starts. In step S1202, first exposure for acorrection coefficient is performed. A readout B image signal is thenstored in the first image memory 501. In step S1203, the lens 101 isdriven to move the focusing lens by a predetermined amount. In stepS1204, second exposure for a correction coefficient is performed. Areadout B image signal is stored in the second image memory 502.

In step S1205, a correction coefficient is calculated. That is, areadout operation is performed for the first image memory 501 and secondimage memory 502, and a predetermined pitch ratio signal is accumulatedin the shading coefficient image memory 508, and read out by thenormalization circuit 510. After that, the signal is normalized to again with a magnification of 1 at the center of the optical axis, andwritten back in the shading coefficient image memory 508.

In step S1206, when the mode is set to a correction mode, the A and Bgain images are output in synchronism with the overall synchronizationsignal. In step S1207, autofocus is performed. At this time, the A and Bimages undergo shading correction by the shading correction units 106and 107, and are input to the focus detection unit 110. Themicrocomputer 109 reads out the result of the focus detection unit 110,and drives the focusing lens to perform focusing.

In step S1208, exposure is performed. In step S1209, it is determinedwhether the mode is a recording mode, thereby determining whether tobranch to step S1207, branch to step S1210 to perform recording, orbranch to step S1211 to terminate the process.

Although not shown, an operation member is connected to themicrocomputer 109, and an instruction from the user can be known.Repeating the process from step S1207 to step S1210 can record an imagewhile performing focus detection.

Second Embodiment

An image capturing apparatus according to the second embodiment of thepresent invention will be described below with reference to FIGS. 13A to13D. FIGS. 13A to 13D are views showing the pixel structure of an imagesensor according to the second embodiment. A photoelectric conversionportion is divided into four photoelectric conversion portions 1401,1402, 1403, and 1404 in correspondence with a microlens 201. By usingthese pixels grouped as indicated by dotted lines in FIG. 13C, it ispossible to obtain an A image by adding the signals of the photoelectricconversion portions 1401 and 1403, and obtain a B image by adding thesignals of the photoelectric conversion portions 1402 and 1404.

Furthermore, as shown in FIG. 13D, a C image signal is obtained byadding the signals of the photoelectric conversion portions 1401 and1402 and a D image signal is obtained by adding the signals of thephotoelectric conversion portions 1403 and 1404. As described above,dividing the photoelectric conversion portion into the four portionsmakes it possible to generate signals in an arrangement in which pupildivision is performed in the vertical direction and in an arrangement inwhich pupil division is performed in the horizontal direction.

FIG. 14 is a block diagram showing the arrangement of the imagecapturing apparatus according to the second embodiment. An ABCD imageaddition separation unit 1501 generates the above-described A, B, C, andD images. The A and B images are processed in the same manner as that inthe first embodiment. Since the vertically pupil-divided signals arenewly generated in the C and D images, a new shading correctioncoefficient calculation unit 1504 is added. Since a vertical image shiftoccurs, an internal delay occurs for each line to correspond to thevertical shift. Vertically pupil-divided images and horizontallypupil-divided images are input to a focus detection unit 110, therebyallowing focus detection for a wider variety of objects.

Although the embodiments of the present invention have been described,the present invention is not limited to them, and various modificationsand changes can be made within the spirit and scope of the presentinvention.

In the first and second embodiments, for example, a case in which aplurality of photodiodes are used to perform pupil division with respectto one microlens has been explained. However, as long as pupil-dividedsignals are used, it is possible to obtain the same effects for signalspupil-divided by blocking light between the microlens and photodiodes.

In addition, in the first and second embodiments, the shadingcoefficient of the A image is obtained by inverting the shadingcoefficient of the B image. However, these coefficients may beseparately calculated.

In the second embodiment, the signals having undergone horizontalshading correction are added to obtain a Bayer array. However, it isdesirable to obtain a Bayer array by adding signals which have beencorrected by both the vertical and horizontal shading coefficients,since it is possible to obtain a better video signal.

Although the first and second embodiments assume an image obtained by asingle-chip image sensor, it is also possible to obtain the effects ofthe present invention using a two- or three-chip image sensor. Aposition where pupil division is performed need not be between themicrolens and the photodiodes.

FIG. 15 shows the optical structure of a system which separates incidentlight into left and right images at the same time in a collimated lightarea (an area where light spread from a point light source at anobject-side focus position is collimated) of a relay lens, and exposes Aand B images using different image sensors. Reference numeral 1301denotes mirrors; 1302, a stop; and 1303, a relay lens. The mirrors 1301divide light collimated by the relay lens 1303 into left and right lightbeams. The divided light beams are reflected by mirrors 1305 and 1304 tobe guided to imaging lenses 1306 and 1307, thereby forming images onimage sensors 1308 and 1309, respectively. This arrangement makes itpossible to simultaneously obtain the A and B images using the two imagesensors. It is thus possible to obtain the same effects as those in thefirst and second embodiments. The A and B images may betime-divisionally exposed.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2013-116245, filed May 31, 2013, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image capturing apparatus comprising: an imagesensor including a photoelectric conversion portion configured toreceive light beams having passed through a portion of a pupil area ofan imaging lens and output an image signal; a control unit configured tooutput, from said photoelectric conversion portion, a plurality of imagesignals captured by shifting a focus position of the imaging lens; anacquisition unit configured to acquire an image shift amount on an imagesensing plane of said image sensor, which corresponds to a shift amountwhen shifting the focus position of the imaging lens; and a calculationunit configured to calculate a correction coefficient for shading on theimage sensing plane based on the image shift amount by comparing signalscorresponding to the same object among the plurality of image signals.2. The apparatus according to claim 1, wherein said calculation unitcompares the signals corresponding to the same object after applying alow-pass filter to the signals of the same object.
 3. The apparatusaccording to claim 1, wherein said calculation unit calculates theshading correction coefficient at each image height by specifying, basedon the image shift amount, the signals corresponding to the same objectfrom two signals at different positions on an optical axis of a focusinglens, and obtaining a ratio between the signals for each image height onthe image sensing plane.
 4. The apparatus according to claim 1, whereinsaid image sensor includes, as the photoelectric conversion portion, afirst photoelectric conversion portion configured to receive lighthaving passed through a first pupil area of the imaging lens, and asecond photoelectric conversion portion configured to receive lighthaving passed through a second pupil area of the imaging lens.
 5. Theapparatus according to claim 4, further comprising a focus detectionunit configured to, after correcting a signal obtained from said firstphotoelectric conversion portion and a signal obtained from said secondphotoelectric conversion portion using the shading correctioncoefficient, calculate a defocus amount based on a shift amount betweenthe signal obtained from said first photoelectric conversion portion andthe signal obtained from said second photoelectric conversion portion.6. The apparatus according to claim 4, further comprising an additionunit configured to, after correcting a signal obtained from said firstphotoelectric conversion portion and a signal obtained from said secondphotoelectric conversion portion using the shading correctioncoefficient, add the signals to obtain a video signal.
 7. A controlmethod for an image capturing apparatus which includes an image sensorhaving a photoelectric conversion portion configured to receive lightbeams having passed through a portion of a pupil area of an imaging lensand output an image signal, the method comprising the steps of: causinga control unit to output, from the photoelectric conversion portion, aplurality of image signals captured by shifting a focus position of theimaging lens; causing an acquisition unit to acquire an image shiftamount on an image sensing plane of the image sensor, which correspondsto a shift amount when shifting the focus position of the imaging lens;and causing a calculation unit to calculate a correction coefficient forshading on the image sensing plane based on the image shift amount bycomparing signals corresponding to the same object among the pluralityof image signals.